PATENT 

IN THE CLAIMS : 

All pending claims are reproduced below. 

1 . (previously presented) A method for generating one or. more audio elements, the 
method comprising the steps of: 

receiving user input; 

creating a first audio element, with an audio data field, in response to the user input; 

displaying a visual representation of the first audio element located spatially on a two 
dimensional layout capable of displaying non-audio elements, wherein a display 
position of the visual representation is independent of the audio data field in the first 
audio element and controllable by a user. 

2. (original) The method of claim 1, wherein the step of receiving user input includes 
receiving control signals, and wherein the step of creating a first audio element creates a data 
structure having a unique identification number field, and a spatial location field. 

3. (previously presented) The method of claim 1, wherein the step of receiving user 
input includes receiving an audio input signal, and wherein the step of creating the first audio 
element includes storing representations of the audio input signal in the audio data field of the 
first audio element. 

4. (original) The method of claim 3, wherein the step of displaying includes 
displaying an audio indicator within the visual representation of the first audio element, the audio 
indicator corresponding to the audio input signal. 

5. (original) The method of claim 4, wherein the step of displaying increases the size 
of the visual representation of the first audio element and the audio indicator based upon a 
duration of the audio input signal. 
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6. (original) The method of claim 4, wherein the audio indicator is a waveform, a 
length of the waveform relating to a time measurement of the audio input signal and the height of 
the waveform relating to a measurement of energy or pitch in the audio input signal. 

7. (original) The method of claim 4, wherein the audio indicator is a color graph, 
colors in the graph relating to a measurement of energy or pitch in the audio input signal. 

8. (original) The method of claim 3, wherein the step of displaying includes 
providing an audio gauge indicating the amplitude of the audio input signal. 

9. (original) The method of claim 3, wherein step of displaying includes displaying a 
control button, the control button beginning the logging of the audio input signal if the audio 
input signal is not being logged, and the control button ending the logging of the audio input 
signal if the signal is being logged. 

10. (original) The method of claim 3, wherein the step of receiving user input includes 
receiving a signal indicating an ordinal value, and wherein the step of creating a first audio 
element includes storing the ordinal value in an ordinal field of the first audio element. 

1 1 . (original) The method of claim 1, further comprising the steps of: 
receiving additional user input; 

creating a second audio element in response to the additional user input; 

displaying a visual representation of the second audio element located spatially on a two 
dimensional layout, wherein the display position of the visual representation of the 
second audio element is independent of a temporal sequence or relationship to the 
first audio element. 

12. (original) The method of claim 11, further comprising the steps of: 
creating a link between the first audio element and the second audio element; and 
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displaying a visual representation of the link as a connector between the visual 

representation of the first audio element and the visual representation of the second 
audio element. 

13. (original) The method of claim 12, wherein the step of creating a link includes the 
steps of: 

creating a link object; 

storing a link to the first audio element and an interval within the first audio element in 

the linking object; and 
storing a link to the second audio element and an interval within the second audio 

element in the linking object. 

14. (original) The method of claim 1, wherein the step of receiving user input 
comprises receiving an identification of a media object while receiving an audio input signal; and 
wherein the method further comprises the step of creating a link between the identified media 
object and the first audio element. 

15. (original) The method of claim 14, wherein the identification of a media object 
includes a range in the media object, and the method further comprise the step of determining a 
range in the first audio element corresponding to when the identification of the media object was 
received, and wherein the step of creating a link stores a reference to the identified media object, 
a reference to the first audio element, the range in the media object and the range in the first 
audio element. 

16. (original) The method of claim 14, wherein the step of displaying includes 
displaying a representation of the link. 

1 7. (original) The method of claim 1 , wherein the step of creating the first audio 
element includes the step of assigning the audio element an ordinal value. 
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1 8. (original) The method of claim 1 7, wherein the step of assigning the audio 
element the ordinal value sets the ordinal value to be one plus the highest ordinal value for an 
existing element. 

19. (original) The method of claim 17, wherein the step of displaying reflects ordinal 
values by showing connecting lines between successive audio elements. 

20. (original) The method of claim 1, further comprising the steps of: 
^ creating a link between the first audio element and a media object; and 

displaying a visual representation of the link as a connector between the visual 
representation of the first audio element and a visual representation of the media 
object. 

21 . (original) The method of claim 20, wherein the step of creating a link includes the 
steps of: 

creating a link object; 

storing a reference to the first audio element and an interval within the first audio element 

in the linking object; and 
storing a reference to the media object and an interval within the media object in the 

linking object. 

22. (previously presented) A method for deleting audio elements, each audio element 
having a graphical component, the method comprising the steps of: 

receiving user input; 

responsive to user input, identifying an audio element to be deleted; 
determining a visual representation including an audio indicator and audio gauge 

corresponding to the identified audio element; 
removing the determined visual representation from display; 
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deleting the identified audio element from memory; 

determining audio elements that have an ordinal value greater than the identified audio 
element; and 

retrieving and decrementing the ordinal value of each audio element determined to have 
an ordinal value greater than the identified audio element. 

23. (previously presented) A method for reordering two or more audio elements, each 
audio element having a graphical component, the method comprising the steps of: 

receiving user input; 

identifying a chosen set of audio elements to be reordered from the user input; 
identifying a new order for the chosen set of audio elements based on the user input such 

that the audio elements are ordered spatially independent of each other; 
determining a lowest ordinal value for the chosen set of audio elements; 
modifying an ordinal value of each of the audio elements in the chosen set of audio 

elements using the new order; and 
updating display of chosen set of audio elements to reflect the modified ordinal values. 

24. (original) The method of claim 23, further comprising the step of storing the 
modified ordinal values for the chosen set of audio elements. 

25. (original) The method of claim 23, wherein the step of determining a lowest 
ordinal value for the chosen set of audio elements further comprising the steps of: 

retrieving each audio element in the chosen set; 

retrieving an ordinal value for each audio element in the chosen set from its audio 
element; and 

comparing the retrieved ordinal values to determine a lowest value. 
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26. (original) The method of claim 23, wherein the user input includes selecting a 
reordering mode and moving a cursor over visual representations corresponding to audio 
elements using a pointing device, and wherein the chosen set is determined by the visual 
representations over which the cursor crosses, and the new order is an order in which the visual 
representations are crossed. 

27. (original) The method of claim 26, wherein the method further comprises a step of 
displaying a feedback as to which visual representations the cursor has been moved across and 
the order in which the cursor was moved across them. 

28. (original) The method of claim 23, further comprising the steps of: 
determining whether there are any audio elements not chosen for reordering; 
if there are audio elements not chosen for reordering, 

identifying any unselected audio elements; 
, determining an ordinal value of each unselected audio element; 

modifying ordinal values of the unselected audio elements that have an ordinal value 

greater than the lowest ordinal value to a modified order; and 
updating display of the unselected audio elements to reflect the modified ordinal values. 

29. (original) A method of associating audio elements with a media object, the 
method comprising the steps of: 

receiving user input identifying a media object and an audio element to be associated; 
creating a link object; 

storing a reference to the media object and the audio element in the link object; and 
displaying a representation of the link object with the representations of the media object 
and the audio element. 
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30. (original) The method of claim 29 further comprising the steps of: 
receiving user input identifying a range in the audio element; and 
storing the range in the link object. 

3 1 . (original) The method of claim 30 wherein the step of displaying further 
comprises: 

displaying a visual representation of the audio element; 
displaying a visual representation of the media object; and 

displaying a connector between the visual representation of the audio element at the range 
and the visual representation of the media object. 

32. (original) The method of claim 29 further comprising the steps of: 
receiving user input identifying a range in the media object; and 
storing the range in the link object. 

33. (original) The method of claim 32 wherein the step of displaying further 
comprises: 

displaying a visual representation of the audio element; 
displaying a visual representation of the media object; and 

displaying a connector between the visual representation of the audio element and the 
range in the visual representation of the media object. 

34. (previously presented) A method of outputting audio signals, the method 
comprising the steps of: 

displaying a visual representation of at least one audio element located spatially on a two- 
dimensional layout capable of displaying non-audio elements , the visual 
representation of the audio element including an audio indicator; 

receiving user input; 
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identifying a first audio element, with an audio data field, based on the user input; 
retrieving first audio data for the identified audio element from the audio data field; 
outputting the first audio data; and 

highlighting sections of the audio indicator corresponding to a temporal range for which 
audio output has been provided. 

35. (original) The method of claim 34, wherein the audio indicator is a waveform, a 
length of the waveform relating to a time measurement of audio data and the height of the 
waveform relating to a measurement of energy or pitch in the audio data. 

36. (original) The method of claim 34, wherein the audio indicator is a color graph, 
colors in the graph relating to a measurement of energy or pitch in the audio data. 

37. (original) The method of claim 34, wherein the highlighting is shading. 

38. (original) The method of claim 34, wherein the audio indicator is a waveform and 
highlighting is rendering the waveform in bold. 

39. (original) The method of claim 34, wherein the audio indicator is a waveform and 
the highlighting is rendering the waveform in a different color from the visual representation of 
the first audio element. 

40. (previously presented) The method- of claim 34, wherein the step of displaying 
includes displaying a plurality of audio elements, and wherein the method further comprises the 
steps of: 

determining an ordinal number for the first audio element; 

determining whether there are any audio elements with an ordinal number greater than the 

ordinal number for the first audio element; 
identifying a second audio element with an ordinal number greater that the ordinal 

number for the first audio element; 
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retrieving second audio data for the second audio element; 
outputting the second audio data for the second audio element; and 
highlighting sections of a waveform corresponding to a temporal range for which audio 
output has been provided. 

41. (previously presented) The method of claim 40, wherein the steps of retrieving 
the second audio data and outputting the second audio data are performed after the first audio 
data has been output. 

42. (previously presented) The method of claim 34, further comprising the steps of: 
receiving a stop signal from the user; and 

terminating the outputting of the first audio data. 

43. (previously presented) The method of claim 34, further comprising the steps of: 
receiving a create new audio element signal from a user; and 

terminating the outputting of the first_audio data. 

44. (original) A method of joining audio elements comprising: 
receiving user input identifying a first and second audio elements to be joined; 
creating a new audio element; 

retrieving information from the first audio element and storing it in the new audio 
element; 

retrieving information from the second audio element and storing it in the new audio 

element; and 
deleting the first and second audio elements. 

45. (original) The method of claim 44, wherein the information from the first audio 
element includes an ordinal value. 
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46. (original) The method of claim 45, further comprising the step of determining the 
ordinal value of the first audio element and using the determined ordinal value as the ordinal 
value for the new audio element. 

47. (original) The method of claim 45, further comprising the step of reducing by one 
the ordinal value of any audio elements having ordinal values higher than that of the first audio 
element. 

48. (previously presented) The method of claim 44, further comprising the step of 
modifying a link which is associated with the second audio element to be associated with the new 
audio element. 

49. (previously presented) The method of claim 48, wherein the step of modifying the 
link includes modifying a reference to point to the new audio element and adjusting the range 
information. 

50. (original) A method of joining audio elements comprising: 
receiving user input identifying a first and second audio elements to be joined; 
retrieving information from the second audio element; 

storing the retrieved information from the second audio element in the first audio 

element; and 
deleting the second audio element. 

51. (original) The method of claim 50, wherein the information from the second audio 
element includes an audio data signal. 

52. (original) The method of claim 50, further comprising the step of reducing by one 
the ordinal value of any audio elements having ordinal values higher than that of the first audio 
element. 
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53. (previously presented) The method of claim 50, further comprising the step of 
modifying a link that is associated with the second audio element to be associated with the first 
audio element. 

54. (previously presented) The method of claim 53, wherein the step of modifying a 
links includes modifying a reference to point to the first audio element and adjusting range 
information. 

55. (previously presented) A method of splitting an audio element, the method 
comprising the steps of: 

receiving user input identifying an original audio element to be split, the original audio 
element containing a beginning point, and_an ending point, and a splitting point, the 
splitting point situated in between the beginning point and the ending point; 

creating a first audio element; 

creating a second audio element; 

retrieving first audio data from the original audio element, the first audio data retrieved 

between the beginning point and the splitting point; 
retrieving second audio data from the original audio element, the second audio data 

retrieved between the splitting point and the ending point; 
storing first audio data in the first audio element; 
storing second audio data in the second audio element; and 
deleting the original audio element. 

56. (previously presented) The method of claim 55, further comprising the steps of 
determining any links to audio data from the splitting point in the original audio element 

to the ending point; 
updating a link object to refer to second audio element; and 
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updating display of the visual representations of the first audio element, the second audio 
element and the link to reflect the new association. 

57. (previously presented) A method of splitting an audio element, the method 
comprising the steps of: 

receiving user input identifying an original audio element to be split, the original audio 
element containing a beginning point, and_an ending point, and a splitting point 
situated in between the beginning point and the ending point; 

creating a new audio element; 

retrieving audio data from the original element, the audio data retrieved between the 

splitting point and the ending point of the original audio element; 
storing the retrieved audio data in the new audio element; and 
deleting retrieved_audio data from the original audio element. 

58. (previously presented) The method of claim 57, further comprising a step of 
displaying visual representations of the original audio element and the new audio element located 
spatially on a two dimensional layout. 

59. (previously presented) The method of claim 57, further comprising the steps of: 
determining an ordinal value for the original audio element; 

storing a value of the determined ordinal value plus one as the ordinal value for the 

second audio element; and 
updating the display of the visual representations of the original and new audio elements 

to reflect the stored ordinal values. 

60. (previously presented) The method of claim 59, further comprising the steps of: 
determining any audio elements having an ordinal value greater than the ordinal value for 

the original audio element; and 
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increasing by one the ordinal values of those determined elements having ordinal values 
higher than that of the original element. 

61 . (previously presented) The method of claim 57, further comprising the steps of: 
determining any links to audio data from the splitting point in the original audio element 

to the ending point; 
updating a link object to refer to the new audio object; and 

updating the display of the visual representations of the original audio element, the new 
audio element and the links to reflect the new association. 

62. (previously presented) A method for displaying media objects in conjunction with 
outputting audio data, the method comprising the steps of: 

identifying an audio element; 

retrieving audio data for the identified audio element from an audio data field; 
outputting the retrieved audio data; 

retrieving a link object referring to the identified audio element; 
retrieving a media object referred to by the link object; and 
displaying the media object. 

63. (previously presented) The method of claim 62, wherein the link object identifies 
an audio range in the audio data field of the audio element to which it refers, and wherein the 
step of displaying is performed when an audio range specified in the link object is reached. 

64. (original) The method of claim 62, wherein the link object identifies a range in the 
media object to which it refers, and wherein the step of displaying, displays the range in the 
media object. 
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65. (original) The method of claim 62, wherein the link object identifies a range in the 
media object to which it refers, and wherein the step of displaying, displays the range in a 
separate window. 
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